Image information conversion apparatus and image information conversion method

ABSTRACT

The system provides an image information conversion apparatus and method by which picture quality deterioration caused by conversion from inputted MPEG2 image compression information into MPEG4 image compression information to be outputted is prevented. When an I picture of an MPEG2 bit stream is to be converted into a P-VOP of an MPEG4 bit stream based on an estimated value of the complexity for each VOP, a scene change detection section detects whether or not the I picture includes a scene change. If a scene change is detected by the scene change detection section, then a GOV structure determination section determines that conversion of the I picture of the MPEG2 bit stream into a P-VOP of an MPEG4 bit stream should not be performed.

BACKGROUND OF THE INVENTION

This invention relates to an image information conversion apparatus andan image information conversion method, and more particularly to animage information conversion apparatus and an image informationconversion method which are used to receive, through network media suchas a satellite broadcast, a cable television broadcast or the Internetor process, on a recording medium such as an optical disk or amagneto-optical disk, image information in the form of a bit streamcompressed by orthogonal transform such as discrete cosine transform andmotion compensation.

In recent years, an apparatus which complies with a method wherein imageinformation is handled as digital data and the redundancy unique toimage information is utilized to compress image information byorthogonal transform such as, for example, discrete cosine transform andmotion compensation in order to allow transmission and storage ofinformation with a high efficiency has been popularized in both ofinformation distribution from a broadcasting station or the like andinformation reception by ordinary households.

Particularly, the MPEG(Moving Picture Experts Group)2 standardized bythe MPEG is defined as a general purpose image coding system in theISO/IEC 13818-2 and covers both of interlaced scan images andprogressive scan images as well as standard resolution images and highresolution images. Therefore, it is expected that the MPEG2 be used bywide varieties of applications from professional applications toconsumer applications in the future.

Where such an MPEG2 compression system as described above is used,realization of a high compression ratio and a good picture quality canbe anticipated by allocating, to interlaced scan images of a standardresolution having, for example, 720×480 pixels, a code amount(hereinafter referred to as bit rate) of 4 to 8 Mbps or by allocating,to interlaced scan images of a high resolution having, for example,1,920×1,088 pixels, a bit rate of 18 to 22 Mbps.

The MPEG2 is directed to high picture quality coding suitableprincipally for broadcasting, but is not ready for a coding system of abit rate lower than, that is, of a compression ratio higher than, thatof the MPEG1. However, from popularization of portable terminals, it hasbeen expected that the need for a coding system of a higher compressionratio increases in the future. Therefore, the MPEG4 coding system hasbeen standardized, and the image coding system of the MPEG4 was approvedas international standards of the ISO/IEC 14496-2 in December 1998.

In order to process MPEG2 image compression information (hereinafterreferred to as MPEG2 bit stream) coded once so as to be suitable fordigital broadcasting on a portable terminal or the like, it is demandedto convert the MPEG2 bit stream into MPEG4 image compression information(hereinafter referred to as MPEG4 bit stream) of a lower bit rate.

An image information conversion apparatus (transcoder) which satisfiesthe demand is disclosed in Susie J. Wee, John G. Apostlopoulos and NickFeamster, “Field-to-Frame Transcoding with Spatial and TemporalDownsampling”, ICIP '99 (hereinafter referred to as document 1). Theimage information conversion apparatus mentioned is shown in FIG. 5.

Referring to FIG. 5, the image information conversion apparatus 101shown includes a picture type discrimination section 111, an MPEG2 imageinformation (I picture and P picture) decoding section 112, a reductionsection 113, a video memory 114, an MPEG4 image information (I/P-VOP)coding section 115, a motion vector synthesis section 116, and a motionvector detection section 117. It is to be noted that the VOP (VideoObject Plane) in the MPEG4 corresponds to the frame in the MPEG2.

The picture type discrimination section 111 receives data of frames ofMPEG2 image compression information (hereinafter referred to as MPEG2bit stream) of an interlaced scan as an input thereto and discriminateswhether data of each frame is of MPEG2 image information (hereinafterreferred to as I picture and P picture which signify an intra-imagecoded picture and a forward predictive coded picture, respectively) orof a B picture (bi-directionally predicted picture). The picture typediscrimination section 111 outputs only the former data to the MPEG2image information decoding section 112 of the following stage.

The MPEG2 image information decoding section 112 executes processingsimilar to that of an ordinary MPEG2 image information decoding section.However, since data regarding B pictures are discarded by the picturetype discrimination section 111, it is only required for the MPEG2 imageinformation decoding section 112 to have a function of decoding only I/Ppictures.

The reduction section 113 receives pixel values from the MPEG2 imageinformation decoding section 112 and performs processing of reducing thepixel values to ½ in the horizontal direction and discarding data of oneof the first and second fields in the vertical direction while leavingdata of the other field to produce a progressive scan image having asize of ¼ that of the inputted image information.

If the MPEG2 bit stream inputted from the MPEG2 image informationdecoding section 112 represents images compliant with the standards ofthe NTSC (National Television System Committee), that is, interlacedscan images of 720×480 pixels and 30 Hz, then the images after thereduction by the reduction section 113 have a size of 360×240 pixels.However, in order to allow the processing in a unit of a macro blockwhen the MPEG4 image information coding section 115 in a succeedingstage performs coding, the pixel numbers both in the horizontal andvertical directions must be multiples of 16. Accordingly, the reductionsection 113 further performs supplementation or discarding of pixels forsatisfying the requirement. In particular, in the specific casedescribed above, eight lines, for example, at the right end or the leftend in the horizontal direction are discarded so that the image has asize of 352×240 pixels.

The progressive scan image produced by the reduction section 113 isstored into the video memory 114 and then undergoes coding processing bythe MPEG4 image information coding section 115, and is outputted as anMPEG4 bit stream.

Motion vector information in the inputted MPEG2 bit stream is suppliedto the motion vector synthesis section 116, by which it is mapped tomotion vectors for the image information after the reduction.

The motion vector detection section 117 detects motion vectors of a highdegree of accuracy based on the motion vector values synthesized by themotion vector synthesis section 116.

The image information conversion apparatus 101 disclosed in document 1produces an MPEG4 bit stream of progressive scan images having a size of½×½ that of an inputted MPEG2 bit stream. For example, where theinputted MPEG2 bit stream complies with the NTSC standards, the MPEG4bit stream to be outputted has the SIF size (352×240 pixels). The imageinformation conversion apparatus 101 can convert the inputted MPEG2 bitstream also into an image of any other image size, for example, the QSIF(176×112 pixels) size which is a size of approximately ¼×¼ in theexample described above, by modifying the operation of the reductionsection 113.

Further, the image information conversion apparatus 101 performs, as aprocess by the MPEG2 image information decoding section 112, a decodingprocess using all of eighth-order discrete cosine transform coefficientsin the inputted MPEG2 bit stream for the horizontal and verticaldirections or a decoding process using only low-frequency componentsfrom among eighth-order discrete cosine transform coefficients only forthe horizontal direction or for both of the horizontal and verticaldirections thereby to reduce the arithmetic operation amount for thedecoding process and the video memory capacity while suppressing thepicture quality deterioration to the minimum.

In the image information conversion apparatus 101 shown in FIG. 5, thecode amount control of the MPEG4 image information coding section 115makes a significant factor of determination of the picture quality of anMPEG4 bit stream. In the ISO/IEC 14496-2, the system for code amountcontrol is not specifically prescribed, and each vendor can use a systemwhich is considered optimum from the point of view of the arithmeticoperation amount and the output picture quality in accordance with anapplication to be used. In the following, a system prescribed in theMPEG2 Test Mode 15 (ISO/IEC JTC1/SC29/WG11 N0400) as a representativecode amount control system is described.

For the code amount control, bit distribution to each picture isperformed as a first step using a target code amount (target bit rate)and a GOP (Group of Pictures) configuration as input variables. The GOPsignifies a group of a plurality of pictures of different types arrayedin accordance with certain specifications. Then, rate control isperformed using a virtual buffer, whereafter adaptive quantization foreach macro block is performed finally taking a visual characteristicinto consideration. The operation of the code amount control isillustrated in FIG. 6.

Referring to FIG. 6, first in step S101, the MPEG4 image informationcoding section 115 distributes an allocation bit amount for each picturein a GOP in accordance with a bit amount (hereinafter represented by R)to be allocated to those pictures which are not decoded as yet includingallocation object pictures. This distribution is repeated in order ofcoded pictures in the GOP. In this instance, the code amount allocationto each picture is performed based on the following two assumptions.

First, it is assumed that the product of an average quantization scalecode to be used for coding of each picture and the generated code amountis fixed for each picture type unless the screen does not change.Therefore, after each picture is coded, variables X_(i), X_(p) and X_(b)(global complexity measures) each representative of the complexity ofthe screen are updated in accordance with the following expressions (1)to (3) for individual picture types:X _(i) =S _(i) ·Q _(i)  (1)X _(p) =S _(p) ·Q _(p)  (2)X _(b) =S _(b) ·Q _(b)  (3)where S_(i), S_(p) and S_(b) are the generated code bit amounts uponpicture coding, and Q_(i), Q_(p) and Q_(b) are average quantizationscale codes upon picture coding. The variables X_(i), X_(p) and X_(b)have initial values represented by the following expressions (4) to (6),respectively, using the target code amount (target bit rate) bit_rate[bits/sec]:X _(i)=160×bit_rate/115  (4)X _(p)=60×bit_rate/115  (5) X _(b)=42×bit_rate/115  (6)

Secondly, it is assumed that the picture quality of the entire image isalways optimized when the ratios K_(p) and K_(b) of the quantizationscale code of P and B pictures with reference to the quantization scalecode of an I picture have values defined by the following expression(7):K _(p)=1.0; K _(b)=1.4  (7)

In particular, the quantization scale code of a B picture is always 1.4times that of the quantization scale codes of I and P pictures. Here, itis supposed that, by coding a B picture rather roughly than I and Ppictures, if the code amount saved with a B picture is added to that ofan I or P picture, then the picture quality of the I or P picture isimproved, and also the picture quality of a B picture which refers tothe I or P picture is improved.

From the two assumptions specified as above, the allocation bit amounts(T_(i), T_(p), T_(b) ) to the different pictures of the GOP have valuesgiven by the following expressions (8) to (10), respectively:$\begin{matrix}{T_{i} = {\max\left\{ {\frac{R}{1 + \frac{N_{p} \cdot X_{p}}{X_{i} \cdot K_{p}} + \frac{N_{b} \cdot X_{b}}{X_{i} \cdot K_{b}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (8) \\{T_{p} = {\max\left\{ {\frac{R}{N_{p} + \frac{N_{b} \cdot K_{p} \cdot X_{b}}{K_{b} \cdot X_{p}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (9) \\{T_{b} = {\max\left\{ {\frac{R}{N_{b} + \frac{N_{p} \cdot K_{b} \cdot X_{p}}{K_{p} \cdot X_{b}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (10)\end{matrix}$where N_(p) and N_(b) are the numbers of P and B pictures which are notcoded in the GOP as yet.

Based on the allocated code amounts determined in this manner, each timea picture is coded in steps S101 and S102, the bit amount R to beallocated to a non-coded picture in the GOP is updated in accordancewith the following expression (11):R=R−S _(i,p,b)  (11)

On the other hand, when the first picture in the GOP is to be coded, thebit amount R is updated in accordance with the following expression(12): $\begin{matrix}{R = {\frac{{bit\_ rate} \times N}{picture\_ rate} + R}} & (12)\end{matrix}$where N is the number of pictures in the GOP. The initial value of thebit amount R at the start of a sequence is 0.

In step S102, in order to make the allocation bit amounts (T_(i), T_(p),T_(b)) to the pictures determined in accordance with the expressions (8)to (10) in step S101 and actual generation code amounts coincide witheach other, quantization scale codes are determined based on capacitiesof three different virtual buffers set independently of each other forthe individual pictures by feedback control in a unit of a macro block.First, prior to code of a j-th macro block, the occupation amounts ofthe virtual buffers are determined in accordance with the followingexpressions (13) to (15): $\begin{matrix}{d_{j}^{i} = {d_{0}^{i} + B_{j - 1} - \frac{T_{i} \times \left( {j - 1} \right)}{MB\_ cnt}}} & (13) \\{d_{j}^{p} = {d_{0}^{p} + B_{j - 1} - \frac{T_{p} \times \left( {j - 1} \right)}{MB\_ cnt}}} & (14) \\{d_{j}^{b} = {d_{0}^{b} + B_{j - 1} - \frac{T_{b} \times \left( {j - 1} \right)}{MB\_ cnt}}} & (15)\end{matrix}$where d₀ ^(i), d₀ ^(p) and d₀ ^(b) are the initial occupation amounts ofthe virtual buffers, B_(j) is the generation bit amount from the top ofthe picture to the j-th macro block, and MB_cnt is the number of macroblocks in 1 picture. The occupation amounts (d_(MB—cnt) ^(i), d_(MB—cnt)^(p), d_(MB—cnt) ^(b)) of the virtual buffers upon ending of coding ofthe individual pictures are used as initial values (d₀ ^(i), d₀ ^(p), d₀^(b)) for the virtual buffer occupations for the next pictures.

Then, the quantization scale code Q_(j) for the j-th macro block iscalculated in accordance with the following expression (16):$\begin{matrix}{Q_{j} = \frac{d_{j} \times 31}{r}} & (16)\end{matrix}$where r is a variable called reaction parameter used to control theresponse of a feedback loop and given by the following expression (17):$\begin{matrix}{r = {2 \times \frac{bit\_ rate}{picture\_ rate}}} & (17)\end{matrix}$

The initial values of the virtual buffers at the start of coding aregiven by the following expressions (18) to (20): $\begin{matrix}{d_{0}^{i} = {10 \times \frac{r}{31}}} & (18)\end{matrix}$  d₀ ^(p) =k _(p) ·d ₀ ^(i)  (19)d₀ ^(b) =K _(b) ·d ₀ ^(i)  (20)

In step S103, the quantization scale codes determined in step S102 aremodified with a variable called activity for each macro block so thatthey may be quantized finely at a flat portion at which deteriorationcan be visually observed comparatively conspicuously but may bequantized roughly at a complicated pattern portion at whichdeterioration can be visually observed comparatively less conspicuously.

The activity is given by the following expression (21) using pixelvalues of totaling 8 blocks including 4 blocks of a frame discretecosine transform mode and 4 blocks of a field discrete cosine transformmode using brightness signal pixel values of the original picture:$\begin{matrix}{{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},8}({var\_ sblk})}}}{{var\_ sblk} = {\frac{1}{64}{\sum\limits_{k = 1}^{64}\quad\left( {P_{k} - \overset{\_}{P}} \right)^{2}}}}{\overset{\_}{p} = {\frac{1}{64}{\sum\limits_{k = 1}^{64}\quad P_{k}}}}} & (21)\end{matrix}$where P_(k) is the brightness signal intra-block pixel value of theoriginal image. The reason why a minimum value is taken in theexpression (21) is that it is intended to use finer quantization where aflat portion is included only at a portion in the macro block.

Further, a normalized activity Nact_(j) whose value ranges from 0.5 to 2is determined in accordance with the following expression (22):$\begin{matrix}{{Nact}_{j} = \frac{{2 \times {act}_{j}} + {avg\_ act}}{{act}_{j} + {2 \times {avg\_ act}}}} & (22)\end{matrix}$where avg-act is the average value of the activity act_(j) of thepicture coded last.

A quantization scale code mquant_(j) with a visual characteristic takeninto consideration is determined in accordance with the followingexpression (23) based on the quantization scale code Q_(j) determined instep S102:

 mquant_(j) =Q _(j) ×N_act_(j)  (23)

By the way, as recited in “Theoretical Analysis of the MPEG CompressionEfficiency and Application thereof to the Code Amount Control”, ShingakuGiho, IE-95, DSP95-10, May 1995 (hereinafter referred to as document 2),the code amount control system defined in the MPEG2 Test Mode 15 doesnot always provide a good picture quality in an MPEG2 image codingsection.

In document 2, the following system is proposed particularly as atechnique for providing an optimum code amount distribution for each offrames in a GOP.

Where N_(I), N_(P) and N_(B) are the numbers of those I, P and Bpictures in a GOP which are not coded as yet and the code amounts to beapplied to them are represented by R_(I), R_(P) and R_(B), respectively,such a fixed rate condition as given by the following expression (24) issatisfied:R=N _(I) ·R _(I) +N _(P) ·R _(P) +N _(B) ·R _(B)  (24)

Where the quantization step sizes of individual frames are representedby Q_(I), Q_(P) and Q_(B) and m is an order number for coordinating aquantization step size and a reproduction error variance with eachother, that is, if it is assumed that minimization of an average of thequantization step sizes raised to the m-th power minimizes thereproduction error variance, then an optimum code amount distributionfor each frame in the GOP is given by minimizing the expression (25)given below: $\begin{matrix}\frac{{N_{I} \cdot Q_{I}^{m}} + {N_{P} \cdot Q_{P}^{m}} + {N_{B} \cdot Q_{B}^{m}}}{N_{I} + N_{P} + N_{B}} & (25)\end{matrix}$

It is to be noted that the average scale Q and the code amount R of theframes are coordinated with the complexity X of each frame as a mediumvariable used also in the MPEG2 Test Mode 15 as given by the followingexpression (26):Q·R ^(a) =X  (26)

Accordingly, by calculating such code amounts R_(I), R_(P) and R_(B) asminimize the expression (25) using the Lagrange's method of undeterminedmultipliers taking the expression (26) into consideration under therestrictive condition of the expression (24), such values as given bythe following expressions (27) to (29) are determined as optimum codeamounts R_(I), R_(P) and R_(B), respectively: $\begin{matrix}{R_{I} = \frac{R}{1 + {N_{P} \cdot \left( \frac{X_{p}}{X_{I}} \right)^{\frac{m}{1 + {m\quad\alpha}}}} + {N_{B} \cdot \left( \frac{X_{B}}{X_{I}} \right)^{\frac{m}{1 + {m\quad\alpha}}}}}} & (27) \\{R_{p} = \frac{R}{N_{P} + {N_{B} \cdot \left( \frac{X_{B}}{X_{P}} \right)^{\frac{m}{1 + {m\quad\alpha}}}}}} & (28) \\{R_{B} = \frac{R}{N_{B} + {N_{P} \cdot \left( \frac{X_{P}}{X_{B}} \right)^{\frac{m}{1 + {m\quad\alpha}}}}}} & (29)\end{matrix}$

Where a α=1, the expressions (27) to (29) and the expressions (8) to(10) given hereinabove in the code amount control system defined in theMPEG2 Test Mode 15 have the following relationship. In particular, fromthe expressions (27) to (29), the parameters K_(p) and K_(b) for codeamount control are adaptively calculated in accordance with thefollowing expression (30) based on the complexities X_(I), X_(P) andX_(B) of each frame: $\begin{matrix}{{K_{p} = \left( \frac{X_{I}}{X_{P}} \right)^{\frac{1}{1 + m}}};{K_{b} = \left( \frac{X_{I}}{X_{B}} \right)^{\frac{1}{1 + m}}}} & (30)\end{matrix}$

In document 2, it is disclosed that a good picture quality is obtainedby setting the value of 1/(1+m) of the expression above to 0.6 to 1.2.

However, when the image information conversion apparatus 101 describedabove with reference to FIG. 5 performs code amount control using thetechnique defined in the MPEG2 Test Mode 15, since it cannot cope with avariation in complexity which is caused by a scene change or the likeoccurring in a GOP, it is difficult to perform the code amount controlstably, which sometimes results in picture quality deterioration.

Thus, another image information conversion apparatus is proposed and isshown in FIG. 7. Referring to FIG. 7, the image information conversionapparatus 102 shown includes, in addition to the components of the imageinformation conversion apparatus 101 described hereinabove withreference to FIG. 5, a compression information analysis section 118, aninformation buffer 119, a complexity calculation section 120 and anMPEG4 image information coding section 121. Detailed description of thecommon components to those of the image information conversion apparatus101 of FIG. 5 is omitted herein to avoid redundancy.

The compression information analysis section 118 analyzes an averagevalue Q over an entire frame of the quantization scale used for decodingprocessing and a total code amount (bit number) B allocated to the framein the MPEG2 bit stream and sends necessary information to theinformation buffer 119.

The information buffer 119 stores such generated code amounts (bitnumbers) and average quantization scales of I/P pictures of the MPEG2bit stream.

The complexity calculation section 120 calculates an estimated value ofthe complexity X for each VOP of MPEG4 image compression information(hereinafter referred to as MPEG4 bit stream) from the information Q andB of each frame stored in the information buffer 119 in accordance withthe expression (20) given hereinabove.

The average value Q over the entire frame of the quantization scale usedfor the decoding processing by the compression information analysissection 118 and the total code amount (bit number) B allocated to theframe in the MPEG2 bit stream are stored into the information buffer119.

The complexity calculation section 120 calculates the complexity X ofeach frame stored in the information buffer 119 from the information Qand B for the frame in accordance with the following expression (31):X=Q·B  (31)

The complexities X of the frames calculated in accordance with theexpression (31) above are buffered for one GOV and then sent as aparameter for code amount control to the MPEG4 image information codingsection 121. Therefore, a delay for one GOV is required. This delay isimplemented using the video memory 114 serving as a delay buffer.

In the following, description is given of in what manner the complexityX of each frame in the GOV calculated in accordance with the expression(31) is used by the MPEG4 image information coding section 121. It is tobe noted that, in the following description, also a case wherein theapparatus does not include the picture type discrimination section 111and does not perform conversion of the frame rate is taken intoconsideration.

The parameters K_(p) and K_(b) determined in accordance with theexpression (30) represent that the ratios of ideal quantization scalesQ_(p—ideal) and Q_(b—ideal) for a P-VOP/B-VOP to an ideal averagequantization scale Q_(1—ideal) for an I-VOP are given by the followingexpression (32): $\begin{matrix}{{\frac{Q_{p\_ ideal}}{Q_{i\_ ideal}} = K_{p}};{\frac{Q_{b\_ ideal}}{Q_{i\_ ideal}} = K_{b}}} & (32)\end{matrix}$

In the MPEG2 Test Mode 15 , the parameters K_(p) and K_(b) are notcalculated adaptively as in the expression (30), but such fixed valuesas given by the expression (7) are used therefor.

From the expressions (30) and (32), where the complexities of anarbitrary VOP 1 and another arbitrary VOP 2 are represented by X₁ and X₂and the ideal quantization scales are represented by Q_(1—ideal) andQ_(2—ideal), respectively, then the following expression (33) isobtained: $\begin{matrix}{\frac{Q_{2{\_ ideal}}}{Q_{1{\_ ideal}}} = {\left( \frac{X_{1}}{X_{2}} \right)^{\frac{1}{1 + m}} \equiv {K\left( {X_{1},X_{2}} \right)}}} & (33)\end{matrix}$

However, where it is desired to use fixed values as given by theexpression (7) as in the MPEG2 Test Mode 15 , the following expression(34) should be used in place of the expression (33) above:$\begin{matrix}{{K\left( {X_{1},X_{2}} \right)} \equiv \left\{ \begin{matrix}{K_{p}\left( {{1 = {I - {VOP}}},{2 = {P - {VOP}}}} \right)} \\{K_{b}\left( {{1 = {I - {VOP}}},{2 = {B - {VOP}}}} \right)} \\{\frac{K_{b}}{K_{p}}\left( {{1 = {P - {VOP}}},{2 = {B - {VOP}}}} \right)} \\{\frac{K_{p}}{K_{b}}\left( {{1 = {B - {VOP}}},{2 = {P - {VOP}}}} \right)} \\{1\left( {{when}\quad 1\quad{and}\quad 2\quad{are}\quad{the}\quad{same}\quad{type}\quad{of}\quad{VOP}} \right)}\end{matrix} \right.} & (34)\end{matrix}$

Here, it is assumed that the total code amount (bit number) allocated tonon-coded VOPs in a GOV is represented by R and the total code amount Ris allocated as R₁, R₂, . . . , R_(n) to the VOPs. In this instance, therelational expression given as the following expression (35) issatisfied by the total code amount and the allocated code amounts R₁,R₂, . . . , R_(n):R=R ₁ +R ₂ + . . . R+  (35)

Among the average quantization scale Q_(k), allocated code amount R_(k)and complexity X_(k) of an arbitrary VOP_(k), the relationshiprepresented by the following expression (36) is satisfied:X _(k) =Q _(k) ·R _(k)  (36)

Here, by transforming the expression (35) taking the expression (36)into consideration, the following expression (37) is obtained:$\begin{matrix}\begin{matrix}{R_{1} = {\frac{R}{\frac{R_{1} + R_{2} + \ldots + R_{n}}{R_{1}}} = \frac{R}{1 + \frac{R_{2}}{R_{1}} + \ldots + \frac{R_{n}}{R_{1}}}}} \\{= \frac{R}{1 + {\frac{Q_{1}}{Q_{2}} \cdot \frac{X_{2}}{X_{1}}} + \ldots + {\frac{Q_{1}}{Q_{n}} \cdot \frac{X_{n}}{X_{1}}}}} \\{= \frac{R}{1 + {\frac{1}{K\left( {X_{1},X_{2}} \right)} \cdot \frac{X_{2}}{X_{1}}} + \ldots + {\frac{1}{K\left( {X_{1},X_{n}} \right)} \cdot \frac{X_{n}}{X_{1}}}}}\end{matrix} & (37)\end{matrix}$

Although the value obtained by the expression (33) or the value obtainedby the expression (34) may be used for K(X₁, X₂) in the expression (37),use of the former can achieve a more optimum code amount distributionsuitable for an image.

Thereupon, if the value of 1/(1+m) is set to 1.0, then the necessity forexponential operation is eliminated, and consequently, high speedexecution can be achieved. Further, even where the value of 1/(1+m) isset to a value other than 1.0, high speed execution can be achieved if atable is prepared in advance and referred to to perform exponentialoperation.

While the complexity X_(k) of each VOP according to the expression (37)is obtained by MPEG4 image coding, if it is assumed that the complexityof each frame by MPEG2 image coding and the complexity of each frame byMPEG4 image coding are equal to each other, then if the complexity X_(k)stored in the complexity calculation section 120 is used, then a targetcode amount for the VOP can be calculated in accordance with theexpression (37).

FIG. 8 illustrates a process when the image information conversionapparatus 102 calculates a target code amount.

Referring to FIG. 8, first in step S111, the MPEG2 image informationdecoding section 112 extracts the average quantization scale Q and theallocated code amount (bit number) B of each frame in a GOP.

In step S112, the complexity calculation section 120 calculates thecomplexity X by operation of the product of the average quantizationscale Q and the allocated code amount (bit number) B of each frame inthe GOP.

Then in step S113, the MPEG4 image information coding section 121calculates a target code amount (target bit rate) based on thecomplexity X.

The image information conversion apparatus 102 produces an MPEG4 bitstream of images of a progressive scan having a size of ½×½ of theinputted MPEG2 bit stream. In particular, if the input MPEG2 bit streamcomplies with the NTSC standards, then the MPEG4 bit stream outputtedhas the SIF size (352×240) The image information conversion apparatus102 can change the operation of the reduction section 113 to convert theinput MPEG2 bit stream into images of any other image size, for example,in the example described above, into images of the QSIF (176×112 pixels)which is an image size of approximately ¼×¼.

Further, the image information conversion apparatus 102 performs, asprocessing by the MPEG2 image information decoding section 112, adecoding process using all of eighth-order discrete cosine transformcoefficients in the inputted MPEG2 bit stream in both of the horizontaland vertical directions and a decoding process using only low frequencycomponents of eighth-order discrete cosine transform coefficients onlyin the horizontal direction or in both of the horizontal and verticaldirections thereby to reduce the arithmetic operation amount and thevideo memory capacity involved in decoding processing while suppressingthe picture quality deterioration.

If the image information conversion apparatus 102 shown in FIG. 7 isused for conversion of an MPEG2 bit stream having a GOP structure of,for example, n=15 and m=3, then an MPEG4 bit stream having a GOVstructure of n=5 and m=1 is obtained as an output. Since the MPEG4 bitstream obtained in this manner has a great number of I-VOPs, the codingefficiency is low and a good picture quality is not obtained in somecases. This problem, however, can be solved by converting an image of anI picture in the input MPEG2 bit stream into a P-VOP of the MPEG4 bitstream to develop GOVs.

The image information conversion apparatus 102 performs motion detectionwithin a fixed search range of an image, which originally is an Ipicture and includes no motion vector, based on motion vectors used forthe last P picture immediately preceding to the I picture to calculatemotion vectors with a high degree of accuracy for the corresponding VOPthereby to prevent the image quality deterioration.

Further, if an I picture is converted into a P-VOP, then since theoriginal complexity relates to the I picture, it has an inappropriatevalue as the complexity after the conversion. The image informationconversion apparatus 102, however, solves the problem just described byusing the complexity for the immediately preceding P picture toeliminate image quality deterioration.

However, while the MPEG2 Text Mode 15 assumes that the complexitiesX_(i), X_(p) and X_(b) as variables representative of the degree ofcomplexity of an image of I, P and B pictures in a GOP are fixed, if theMPEG4 image information coding section 115 actually uses the techniquedefined in the MPEG2 Test Mode 15 to perform code amount control, thenthe assumption is not satisfied in such a case that the GOP includes ascene change or the background exhibits a remarkable variation in theGOP, but rather disturbs stabilized code amount control and makes acause of picture quality deterioration.

Conversion of an I picture of an inputted MPEG2 bit stream into a P-VOPof an MPEG4 bit stream is considered here.

FIG. 9 diagrammatically illustrates a manner wherein an I picture of aninputted MPEG2 bit stream is converted into and outputted as a P-VOP ofan MPEG4 bit stream. Referring to FIG. 9, conversion of the second Ipicture I, into a P-VOP is taken as an example. In this instance, as thecomplexity as a parameter for code amount control for the I picture I₁,the complexity X_(P3) of the P picture P₃ immediately preceding to the Ipicture I₁ is applied.

If the I picture I₁ is an image including a scene change, then acomparatively great code amount must be applied to the I picture I₁.However, since the complexity X_(P3) of the P picture P₃ of theimmediately preceding frame is used as, the complexity for the I pictureI₁ as described above, a sufficient code amount is not allocated to theI picture I₁, resulting in deterioration of the picture quality.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an image informationconversion apparatus and an image information conversion method by whichpicture quality deterioration caused by conversion from inputted firstimage compression information into second image compression informationto be outputted is prevented.

In order to attain the object described above, according to an aspect ofthe present invention, there is provided an image information conversionapparatus including: conversion means for converting first imagecompression information inputted to the image information conversionapparatus into second image compression information to be outputted fromthe image information conversion apparatus, each of the first imagecompression information and the second image compression informationincluding at least intra-image coded pictures and inter-image predictivecoded pictures; and scene change detection means operable when theconversion means calculates, based on a variable representative of acomplexity of a screen for each frame of the inputted first imagecompression information, a target code amount for each frame of thesecond image compression information to be outputted for detecting,prior to conversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted.

According to another aspect of the present invention, there is providedan image information conversion method including the steps of:converting inputted first image compression information into secondimage compression information to be outputted, each of the first imagecompression information and the second image compression informationincluding at least intra-image coded pictures and inter-image predictivecoded pictures; and when to calculate, based on a variablerepresentative of a complexity of a screen for each frame of theinputted first image compression information, a target code amount foreach frame of the second image compression information to be outputted,prior to conversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted.

In the image information conversion apparatus and the image informationconversion method, the product of a code amount allocated to each frameand an average quantization scale of the first image compressioninformation may be used as the variable representative of the complexityof the screen for the frame to detect whether or not a scene change isincluded in the frame. When it is detected that a scene change isincluded in the frame to be converted, preferably the conversion fromthe intra-image coded picture into an inter-image predictive codedpicture is limited.

Further, the variable representative of the complexity of the screen ofthe immediately preceding intra-image coded picture may be subtractedfrom the variable representative of the complexity of the screen of theintra-image coded picture of the inputted first image compressioninformation, and it may be determined that a scene change is includedwhen the absolute value of the difference obtained by the subtraction ishigher than a threshold value determined in advance.

With the image information conversion apparatus and the imageinformation conversion method, deterioration of an image involved inconversion of the first image compression information into the secondimage compression information and deterioration of an image involved inconversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information can be prevented.

The above and other objects, features and advantages of the presentinvention will become apparent from the following description and theappended claims, taken in conjunction with the accompanying drawings inwhich like parts or elements denoted by like reference symbols.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an imageinformation conversion apparatus to which the present invention isapplied;

FIG. 2 is a flow chart illustrating operation of the image informationconversion apparatus of FIG. 1 when a scene change detection section anda GOV structure determination section detect a scene change;

FIG. 3 is a diagrammatic view illustrating a manner wherein the imageinformation conversion apparatus of FIG. 1 converts an I picture of aninputted MPEG2 bit stream into a P-VOP of an MPEG4 bit stream to beoutputted;

FIG. 4 is a flow chart illustrating operation of the scene changedetection section and the GOV structure determination section of theimage information conversion apparatus of FIG. 1;

FIG. 5 is a block diagram showing a configuration of a related art imageinformation conversion apparatus;

FIG. 6 is a flow chart illustrating operation of an MPEG4 imageinformation coding section of the image information conversion apparatusof FIG. 5 which performs code amount control using a complexity of eachframe extracted by an MPEG2 image information decoding section;

FIG. 7 is a block diagram showing a configuration of another related artimage information conversion apparatus;

FIG. 8 is a flow chart illustrating operation of the image informationconversion apparatus of FIG. 7 when it calculates a target code amount;and

FIG. 9 is a diagrammatic view illustrating a manner wherein the imageinformation conversion apparatus of FIG. 8 converts an I picture of aninputted MPEG2 bit stream into a P-VOP of an MPEG4 bit stream to beoutputted

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

An image information conversion apparatus according to the presentinvention detects, upon calculation of a target code amount for eachframe of MPEG4 image compression information to be outputted based onthe complexity of each frame of inputted MPEG2 image compressioninformation, whether or not the frame to be converted includes a scenechange prior to conversion from an intra-image coded picture into aninter-image predictive coded picture, and limits, if a scene change isdetected, the conversion from the intra-image coded picture into aninter-image predictive coded picture. Consequently, deterioration of thepicture quality which occurs upon conversion from an intra-image codedpicture into an inter-picture predictive coded picture can be prevented.

Referring to FIG. 1, there is shown an image information conversionapparatus to which the present invention is applied. The imageinformation conversion apparatus 1 shown includes a picture typediscrimination section 11, a compression information analysis section12, a MPEG2 image information decoding section 13, a reduction section14, a video memory 15, an MPEG4 image information coding section 16, amotion vector synthesis section 17, a motion vector detection section18, an information buffer 19, a complexity calculation section 20, ascene change detection section 21, and a GOV structure determinationsection 22.

The picture type discrimination section 11 receives data of frames ofMPEG2 image compression information of an interlaced scan (hereinafterreferred to as MPEG2 bit stream) as an input thereto and discriminatesof which one of an intra-image coded picture (hereinafter referred to asI picture), a forward predictive coded picture (hereinafter referred toas P picture) and a bi-directionally predicted coded picture(hereinafter referred to as B picture) the data of each frame is. Thepicture type discrimination section 11 transmits information regarding Ipictures and P pictures (hereinafter referred to as I/P pictures) to thecompression information analysis section 12 of the following stage butdiscards information regarding B pictures.

The compression information analysis section 12 analyzes a average valueQ over an entire frame of the quantization scale used for decodingprocessing and a total code amount (bit number) B allocated to the framein the MPEG2 bit stream and sends necessary information to theinformation buffer 19.

The information buffer 19 stores such generated code amounts (bitnumbers) and average quantization scales of I/P pictures of the MPEG2bit stream.

The complexity calculation section 20 calculates an estimated value ofthe complexity X for each VOP of MPEG4 image compression information(hereinafter referred to as MPEG4 bit stream) from the information Q andB of each frame stored in the information buffer 19 in accordance withthe expression (38) given below. It is to be noted that the VOP (VideoObject Plane) corresponds to a frame of the MPEG2. $\begin{matrix}{{\frac{Q_{p\_ ideal}}{Q_{i\_ ideal}} = K_{p}};{\frac{Q_{b\_ ideal}}{Q_{i\_ ideal}} = K_{b}}} & (38)\end{matrix}$

The MPEG2 image information decoding section 13 performs decodingprocessing of information regarding I/P pictures of the MPEG2 bitstream. While the MPEG2 image information decoding section 13 is similarto an ordinary MPEG2 image information decoding section, since dataregarding B pictures is discarded by the picture type discriminationsection 11, it is required that the MPEG2 image information decodingsection 13 can decode at least I/P pictures.

The reduction section 14 receives pixel values as an input thereto fromthe MPEG2 image information decoding section 13, performs a reductionprocess to ½ in the horizontal direction for the pixel values and thenperforms a process of discarding data of only one of the first field andthe second field in the vertical direction while leaving data of theother field thereby to produce an image of a progressive scan having asize of ¼ that of the inputted image information.

If the MPEG2 bit stream inputted from the MPEG2 image informationdecoding section 13 represents images conforming with, for example, thestandards of the NTSC (National Television System Committee), that is,interlaced scan images of 30 Hz of 720×480 pixels, then the picture sizeafter the reduction processing by the reduction section 14 is 360×240pixels. However, in order to allow processing to be performed in a unitof a macro block when coding is performed by the MPEG4 image informationcoding section 16 in a following stage, both of the numbers of pixels ofthe image in the horizontal and vertical directions must be multiples of16. Accordingly, the reduction section 14 further performssupplementation or discarding of pixels to satisfy the requirement. Inparticular, in the case described above, form example, 8 lines at theright end or the left end in the horizontal direction are discarded toproduce an image of 352×240 pixels. Here, MPEG4 image information isreferred to as I/P-VOP.

The pictures of a progressive scan produced by the reduction section 14are stored into the video memory 15 and then undergo coding processingby the MPEG4 image information coding section 16, and consequently areoutputted as an MPEG4 bit stream.

Motion vector information in the input MPEG2 bit stream is supplied tothe motion vector synthesis section 17 and mapped to motion vectors ofthe image information after the reduction.

The motion vector detection section 18 detects motion vectors of highaccuracy based on the motion vector values synthesized by the motionvector synthesis section 17.

The image information conversion apparatus 1 produces an MPEG4 bitstream of images of a progressive scan having a size of ½×½ of theinputted MPEG2 bit stream. In particular, if the input MPEG2 bit streamcomplies with, for example, the NTSC standards, then the MPEG4 bitstream outputted has the SIF size (352×240). The image informationconversion apparatus 1 can change the operation of the reduction section14 to convert the input MPEG2 bit stream into images of any other imagesize, for example, in the example described above, into images of theQSIF (176×112 pixels) which is an image size of approximately ¼×¼.

Further, the image information conversion apparatus 1 performs, asprocessing by the MPEG2 image information decoding section 13, adecoding process using all of eighth-order discrete cosine transformcoefficients in the inputted MPEG2 bit stream in both of the horizontaland vertical directions and a decoding process using only low frequencycomponents of eighth-order discrete cosine transform coefficients onlyin the horizontal direction or in both of the horizontal and verticaldirections thereby to reduce the arithmetic operation amount and thevideo memory capacity involved in decoding processing while suppressingthe picture quality deterioration.

The average value Q over the entire frame of the quantization scale usedfor the decoding processing by the compression information analysissection 12 and the total code amount (bit number) B allocated to theframe in the MPEG2 bit stream are stored into the information buffer 19.

The complexity calculation section 20 calculates the complexity X ofeach frame stored in the information buffer 19 from the information Qand B for the frame in accordance with the following expression (39):X=Q·B  (39)

The complexities X of the frames calculated in accordance with theexpression (33) above are buffered for one GOV and then sent as aparameter for code amount control to the MPEG4 image information codingsection 16. Therefore, a delay for one GOV is required. This delay isimplemented using the video memory 15 serving as a delay buffer.

In the following, description is given of in what manner the complexityX of each frame in the GOV calculated in accordance with the expression(39) is used by the MPEG4 image information coding section 16. It is tobe noted that, in the following description, also a case wherein theapparatus does not include the picture type discrimination section 11and does not perform conversion of the frame rate is taken intoconsideration.

The parameters K_(p) and K_(b) determined in accordance with theexpression (40) given below represent that the ratios of idealquantization scales Q_(p—ideal) and Q_(b—ideal) for a P-VOP/B-VOP to anideal average quantization scale Q_(i—ideal) for an I-VOP are given bythe following expression (41): $\begin{matrix}{{K_{p} = \left( \frac{X_{I}}{X_{P}} \right)^{\frac{1}{1 + m}}};{K_{b} = \left( \frac{X_{I}}{X_{B}} \right)^{\frac{1}{1 + m}}}} & (40) \\{{\frac{Q_{p\_ ideal}}{Q_{i\_ ideal}} = K_{p}};{\frac{Q_{b\_ ideal}}{Q_{i\_ ideal}} = K_{b}}} & (41)\end{matrix}$

In the MPEG2 Test Mode 15 , the parameters K_(p) and K_(b) are notcalculated adaptively as in the expression (40), but such fixed valuesas given by the following expression (42) are used therefor:K _(p)=1.0; K _(b)=1.4  (42)

From the expressions (40) and (41), where the complexities of anarbitrary VOP 1 and another arbitrary VOP 2 are represented by X₁ and X₂and the ideal quantization scales are represented by Q_(1—ideal) andQ_(2—ideal), respectively, then the following expression (43) isobtained: $\begin{matrix}{\frac{Q_{2{\_ ideal}}}{Q_{1{\_ ideal}}} = {\left( \frac{X_{1}}{X_{2}} \right)^{\frac{1}{1 + m}} \equiv {K\left( {X_{1},X_{2}} \right)}}} & (43)\end{matrix}$

However, where it is desired to use fixed values as given by theexpression (42) as in the MPEG2 Test Mode 15 , the following expression(44) should be used in place of the expression (43) above:$\begin{matrix}{{K\left( {X_{1},X_{2}} \right)} \equiv \left\{ \begin{matrix}{K_{p}\left( {{1 = {I - {VOP}}},{2 = {P - {VOP}}}} \right)} \\{K_{b}\left( {{1 = {I - {VOP}}},{2 = {B - {VOP}}}} \right)} \\{\frac{K_{b}}{K_{p}}\left( {{1 = {P - {VOP}}},{2 = {B - {VOP}}}} \right)} \\{\frac{K_{p}}{K_{b}}\left( {{1 = {B - {VOP}}},{2 = {P - {VOP}}}} \right)} \\{1\left( {{when}\quad 1\quad{and}\quad 2\quad{are}\quad{the}\quad{same}\quad{type}\quad{of}\quad{VOP}} \right)}\end{matrix} \right.} & (44)\end{matrix}$

Here, it is assumed that, where the total code amount (bit number)allocated to non-coded VOPs in a GOV is represented by R, when the totalcode amount R is allocated as R₁, R₂, . . . R_(n) to the VOPs, thepicture quality of the GOV is optimized. In this instance, therelational expression given as the following expression (45) issatisfied by the total code amount R and the allocated code amounts R₁,R₂, . . . , R_(n):R=R ₁ +R ₂ + . . . +R _(n)  (45)

Among the average quantization scale Q_(k), allocated code amount R_(k)and complexity X_(k) of an arbitrary VOP_(k), the relationshiprepresented by the following expression (46) is satisfied:X _(k) =Q _(k) ·R _(k)  (46)

In the expression (46) above, the allocated code amount R (R_(k)) may bean allocated code amount (bit number) to each entire frame, an allocatedcode amount (bit number) to a brightness signal of each frame, or anallocated code amount to brightness and color difference signals of eachframe. Further, by transforming the expression (45) taking theexpression (46) into consideration, the following expression (47) isobtained: $\begin{matrix}\begin{matrix}{R_{1} = {\frac{R}{\frac{R_{1} + R_{2} + \ldots + R_{n}}{R_{1}}} = \frac{R}{1 + \frac{R_{2}}{R_{1}} + \ldots + \frac{R_{n}}{R_{1}}}}} \\{= \frac{R}{1 + {\frac{Q_{1}}{Q_{2}} \cdot \frac{X_{2}}{X_{1}}} + \ldots + {\frac{Q_{1}}{Q_{n}} \cdot \frac{X_{n}}{X_{1}}}}} \\{= \frac{R}{1 + {\frac{1}{K\left( {X_{1},X_{2}} \right)} \cdot \frac{X_{2}}{X_{1}}} + \ldots + {\frac{1}{K\left( {X_{1},X_{n}} \right)} \cdot \frac{X_{n}}{X_{1}}}}}\end{matrix} & (47)\end{matrix}$

Although the value obtained by the expression (43) or the value obtainedby the expression (44) may be used for K(X₁, X₂) in the expression (47),use of the former can achieve a more optimum code amount distributionsuitable for an image.

Thereupon, if the value of 1/(1+m) is set to 1.0, then the necessity forexponential operation is eliminated, and consequently, high speedexecution can be achieved. Further, even where the value of 1/(1+m) isset to a value other than 1.0, high speed execution can be achieved if atable is prepared in advance and referred to to perform exponentialoperation.

While the complexity X_(k) of each VOP according to the expression (47)is obtained by MPEG4 image coding, if it is assumed that the complexityof each frame by MPEG2 image coding and the complexity of each frame byMPEG4 image coding are equal to each other, by using the complexityX_(k) stored in the complexity calculation section 20, a target codeamount for the VOP can be calculated in accordance with the expression(47).

FIG. 2 illustrates a processing flow when the image informationconversion apparatus 1 calculates a target code amount.

Referring to FIG. 2, first in step S1, the MPEG2 image informationdecoding section 13 extracts the average quantization scale Q and theallocated code amount (bit number) B of each frame in a GOP.

In step S2, the complexity calculation section 20 calculates thecomplexity X.

Then in step S3, the MPEG4 image information coding section 16calculates a target code amount (target bit rate) based on thecomplexity X. Here, estimated values of the complexity for VOPs of theMPEG4 bit stream calculated in accordance with the expression (46) andto be outputted are stored in the complexity calculation section 20.

The scene change detection section 21 detects based on the estimatedvalues of the complexity for the VOPs whether or not an I picture of theinputted MPEG2 bit stream to be converted into a P-VOP of an MPEG4 bitstream which corresponds to a P picture includes a scene change.

FIG. 3 diagrammatically illustrates a manner wherein an I picture of aninputted MPEG2 bit stream is converted into and outputted as a P-VOP ofan MPEG4 bit stream.

Referring to FIG. 3, reference characters I₀ and I₁ denote each an Ipicture of the MPEG2 bit stream, and P₀, P₁, P₂, P₃, P₄ and P₅ denoteeach a P picture of the MPEG2 bit stream. Further, reference charactersX_(I0) and X_(I1) denote each a complexity as a variable representativeof a complexity of a screen of an I picture, and X_(P0), X_(P1) X_(P2),X_(P3), X_(P4) and X_(P5) denote each a complexity as a variablerepresentative of a complexity of a screen of a P picture.

Here, conversion of the second I picture I₁ into a P-VOP of an MPEG4 bitstream is considered. In this instance, if the I picture I₁, includes ascene change, then in order to prevent picture quality deteriorationupon conversion, a comparatively great code amount must be allocated tothe I picture I₁. Therefore, it is first detected whether or not the Ipicture I₁ includes a scene change.

The scene change detection section 21 of the image informationconversion apparatus 1 subtracts, from the complexity X_(I1) of the Ipicture I₁ of the inputted MPEG2 bit stream, the complexity X_(I0) ofthe immediately preceding I picture I₀ of the inputted MPEG2 bit stream,and compares the absolute value of the resulting difference with athreshold value TH determined in advance. Here, it is assumed that ascene change of the I picture I₁ is detected when the comparison revealsthat the absolute value is higher than the predetermined threshold valueTH.

Accordingly, the scene change detection section 21 discriminates that ascene change is included in the I picture I₁ when the expression (48)given below is satisfied:|X _(I1) −X _(I0) |>TH  (48)

If the scene change detection section 21 detects a scene change, thenthe GOV structure determination section 22 determines that conversion ofthe I picture I₁ of the MPEG2 bit stream into a P-VOP of an MPEG4 bitstream should not be performed.

A series of operations of the scene change detection section 21 and theGOV structure determination section 22 is illustrated in FIG. 4.

Referring to FIG. 4, in step S11, the MPEG2 image information decodingsection 13 extracts the average quantization scale Q and the allocatedcode amount (bit number) B of each frame in a GOP.

In step S12, the complexity X of each frame is calculated by operationof the product of the average quantization scale Q and the allocatedcode amount (bit number) B.

In step S13, the MPEG2 image information decoding section 13discriminates whether or not the absolute value of the difference whenit subtracts, from the complexity X_(I1) of the I picture I₁ of theinputted MPEG2 bit stream, the complexity X_(I0) of the immediatelypreceding I picture I₀ of the MPEG2 bit stream is higher than thepredetermined threshold value TH.

If the absolute value of the difference is equal to or lower than thepredetermined threshold value TH, then the GOV structure determinationsection 22 performs conversion from the I picture I₁ into a P-VOP instep S14.

However, if the absolute value of the difference is higher than thepredetermined threshold value TH, then the GOV structure determinationsection 22 does not perform conversion from the I picture I₁ into aP-VOP in step S15.

Accordingly, as described in detail above, when a scene change isdetected by the scene change detection section 21, the GOV structuredetermination section 22 does not perform conversion of an I picture ofan inputted MPEG2 bit stream into a P-VOP of an MPEG4 bit stream whichcorresponds to a P picture. Consequently, picture quality deteriorationwhich occurs upon conversion from an I picture into a P-VOP can beprevented.

The manner of detection of a scene change is not limited to the methodwhich uses the complexity X as in the expression (48). A scene changemay otherwise be detected that a scene change is included, for example,in the I picture I₁ when, where average values of pixel values of the Ipictures I₀ and I₁ illustrated in FIG. 3 are represented by Mean_I₀ andMean_I₁, respectively, the absolute value of the difference between theaverage values Mean_I₀ and Mean_I₁ is higher than a threshold value THdetermined in advance therefor.

In short, presence/absence of a scene change may be detected dependingupon whether or not the following expression (49) is satisfied:|Mean_(—) I ₁−Mean_(—) I ₀ |>TH  (49)

Here, the values Mean_I₀ and Mean_I₁ may not only be average values ofall pixel values but otherwise be average values of DC components ofmacro blocks as a predetermined coding unit over entire frames, averagevalues of the brightness signal component among pixels, or averagevalues of the brightness signal component among pixels and averagevalues of the color difference signal component among pixels.

Further, while the image information conversion apparatus 1 describedabove receives an MPEG2 bit stream as an input thereto and outputs anMPEG4 bit stream, the input to and the output from the image informationconversion apparatus 1 are not limited to the specific bit streams, andbit streams of, for example, the MPEG1 or the H.263 may be used instead.

While a preferred embodiment of the present invention has been describedusing specific terms, such description is for illustrative purposesonly, and it is to be understood that changes and variations may be madewithout departing from the spirit or scope of the following claims.

The entire disclosure of Japanese Patent Application No. 2000-344491filed on Nov. 10, 2000 including specification, claims, drawings andsummary is incorporated herein by reference in its entity.

1. An image information conversion apparatus comprising: conversionmeans for converting first image compression information inputted tosaid image information conversion apparatus into second imagecompression information to be outputted from said image informationconversion apparatus; each of the first image compression informationand the second image compression information including at leastintra-image coded pictures and inter-image predictive coded pictures;and scene change detection means operable when said conversion meanscalculates, based on a variable representative of a complexity of ascreen for each frame of the inputted first image compressioninformation, a target code amount for each frame of the second imagecompression information to be outputted for detecting, prior toconversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when said scene change detection means detects that a scenechange is included in the frame to be converted, said scene changedetection means limits the conversion by said conversion means from theintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, wherein said scene change detection means usesthe product of a code amount allocated to each frame and an averagequantization scale of the first image compression information as thevariable representative of the complexity of the screen for the frame todetect whether or not a scene change is included in the frame, andwherein said scene change detection means subtracts, from the variablerepresentative of the complexity of the screen of the intra-image codedpicture of the inputted first image compression information, thevariable representative of the complexity of the screen of theimmediately preceding intra-image coded picture and determines that ascene change is included when the absolute value of the differenceobtained by the subtraction is higher than a threshold value determinedin advance.
 2. An image information conversion apparatus comprising:conversion means for converting first image compression informationinputted to said image information conversion apparatus into secondimage compression information to be outputted from said imageinformation conversion apparatus; each of the first image compressioninformation and the second image compression information including atleast intra-image coded pictures and inter-image predictive codedpictures; and scene change detection means operable when said conversionmeans calculates, based on a variable representative of a complexity ofa screen for each frame of the inputted first image compressioninformation, a target code amount for each frame of the second imagecompression information to be outputted for detecting, prior toconversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when said scene change detection means detects that a scenechange is included in the frame to be converted, said scene changedetection means limits the conversion by said conversion means from theintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, wherein said scene change detection means usesthe product of a code amount allocated to each frame and an averagequantization scale of the first image compression information as thevariable representative of the complexity of the screen for the frame todetect whether or not a scene change is included in the frame, andwherein the code amount allocated to each frame of the inputted firstimage compression information is an entire code amount allocated to theframe.
 3. An image information conversion apparatus comprising:conversion means for converting first image compression informationinputted to said image information conversion apparatus into secondimage compression information to be outputted from said imageinformation conversion apparatus; each of the first image compressioninformation and the second image compression information including atleast intra-image coded pictures and inter-image predictive codedpictures; and scene change detection means operable when said conversionmeans calculates, based on a variable representative of a complexity ofa screen for each frame of the inputted first image compressioninformation, a target code amount for each frame of the second imagecompression information to be outputted for detecting, prior toconversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when said scene change detection means detects that a scenechange is included in the frame to be converted, said scene changedetection means limits the conversion by said conversion means from theintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, wherein said scene change detection means usesthe product of a code amount allocated to each frame and an averagequantization scale of the first image compression information as thevariable representative of the complexity of the screen for the frame todetect whether or not a scene change is included in the frame, andwherein the code amount allocated to each frame of the inputted firstimage compression information includes a code amount at least of abrightness signal component.
 4. An image information conversionapparatus according to claim 3, wherein the code amount allocated toeach frame of the inputted first image compression information includesa code amount regarding a brightness signal component and a code amountregarding a color difference signal component.
 5. An image informationconversion apparatus comprising: conversion means for converting firstimage compression information inputted to said image informationconversion apparatus into second image compression information to beoutputted from said image information conversion apparatus; each of thefirst image compression information and the second image compressioninformation including at least intra-image coded pictures andinter-image predictive coded pictures; and scene change detection meansoperable when said conversion means calculates, based on a variablerepresentative of a complexity of a screen for each frame of theinputted first image compression information, a target code amount foreach frame of the second image compression information to be outputtedfor detecting, prior to conversion of an intra-image coded picture ofthe first image compression information into an inter-image predictivecoded picture of the second image compression information, whether ornot a scene change is included in a frame of the intra-image codedpicture to be converted, wherein, when said scene change detection meansdetects that a scene change is included in the frame to be converted,said scene change detection means limits the conversion by saidconversion means from the intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, and wherein said scene changedetection means detects whether or not a scene change is included in theframe to be converted using an average value among pixel values of eachframe.
 6. An image information conversion apparatus according to claim5, wherein said scene change detection means subtracts, from the averagevalue among the pixels of the intra-image coded picture, the averagevalue among the pixels of the immediately preceding intra-image codedpicture and determines that a scene change is included when the absolutevalue of the difference obtained by the subtraction is higher than athreshold value determined in advance.
 7. An image informationconversion apparatus according to claim 5, wherein the average value ofthe intra-image coded picture among the pixels includes an average valueat least of a brightness signal component among the pixels.
 8. An imageinformation conversion apparatus according to claim 7, wherein theaverage value of pixels of the intra-image coded picture includes anaverage value of the brightness signal component among the pixels and anaverage value of a color difference signal component among the pixels.9. An image information conversion apparatus comprising: conversionmeans for converting first image compression information inputted tosaid image information conversion apparatus into second imagecompression information to be outputted from said image informationconversion apparatus; each of the first image compression informationand the second image compression information including at leastintra-image coded pictures and inter-image predictive coded pictures;and scene change detection means operable when said conversion meanscalculates, based on a variable representative of a complexity of ascreen for each frame of the inputted first image compressioninformation, a target code amount for each frame of the second imagecompression information to be outputted for detecting, prior toconversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when said scene change detection means detects that a scenechange is included in the frame to be converted, said scene changedetection means limits the conversion by said conversion means from theintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, and wherein said scene change detection meansdetects whether or not a scene change is included in the frame to beconverted using an average value of a DC component among predeterminedcoding units included in each frame.
 10. An image information conversionapparatus according to claim 9, wherein said scene change detectionmeans subtracts, from the average value of the DC component among thepredetermined coding units of the intra-image coded picture, the averagevalue of the DC component among the predetermined coding units of theimmediately preceding intra-image coded picture and determines that ascene change is included when the absolute value of the differenceobtained by the subtraction is higher than a threshold value determinedin advance.
 11. An image information conversion apparatus according toclaim 9, wherein the average value of the DC component among thepredetermined coding units of the intra-image coded picture includes anaverage value of the DC component at least of a brightness signalcomponent among the predetermined coding units.
 12. An image informationconversion apparatus according to claim 11, wherein the average value ofthe DC component among the predetermined coding units of the intra-imagecoded picture includes an average value of the DC component of abrightness signal component among the predetermined coding units and anaverage value of the DC component of a color difference signal componentamong the predetermined coding units.
 13. An image informationconversion method comprising the steps of: converting inputted firstimage compression information into second image compression informationto be outputted, each of the first image compression information and thesecond image compression information including at least intra-imagecoded pictures and inter-image predictive coded pictures; and detecting,when to calculate, based on a variable representative of a complexity ofa screen for each frame of the inputted first image compressioninformation, a target code amount for each frame of the second imagecompression information to be outputted, prior to conversion of anintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, whether or not a scene change is included in aframe of the intra-image coded picture to be converted, wherein, when itis detected that a scene change is included in the frame to beconverted, the conversion from the intra-image coded picture into aninter-image predictive coded picture is limited, wherein the product ofa code amount allocated to each frame and an average quantization scaleof the first image compression information is used as the variablerepresentative of the complexity of the screen for the frame to detectwhether or not a scene change is included in the frame, wherein thevariable representative of the complexity of the screen of theimmediately preceding intra-image coded picture is subtracted from thevariable representative of the complexity of the screen of theintra-image coded picture of the inputted first image compressioninformation, and it is determined that a scene change is included whenthe absolute value of the difference obtained by the subtraction ishigher than a threshold value determined in advance.
 14. An imageinformation conversion method comprising the steps of: convertinginputted first image compression information into second imagecompression information to be outputted, each of the first imagecompression information and the second image compression informationincluding at least intra-image coded pictures and inter-image predictivecoded pictures; and detecting, when to calculate, based on a variablerepresentative of a complexity of a screen for each frame of theinputted first image compression information, a target code amount foreach frame of the second image compression information to be outputted,prior to conversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when it is detected that a scene change is included in theframe to be converted, the conversion from the intra-image coded pictureinto an inter-image predictive coded picture is limited, wherein theproduct of a code amount allocated to each frame and an averagequantization scale of the first image compression information is used asthe variable representative of the complexity of the screen for theframe to detect whether or not a scene change is included in the frame,wherein the code amount allocated to each frame of the inputted firstimage compression information is an entire code amount allocated to theframe.
 15. An image information conversion method comprising the stepsof: converting inputted first image compression information into secondimage compression information to be outputted, each of the first imagecompression information and the second image compression informationincluding at least intra-image coded pictures and inter-image predictivecoded pictures; and detecting, when to calculate, based on a variablerepresentative of a complexity of a screen for each frame of theinputted first image compression information, a target code amount foreach frame of the second image compression information to be outputted,prior to conversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when it is detected that a scene change is included in theframe to be converted, the conversion from the intra-image coded pictureinto an inter-image predictive coded picture is limited, wherein theproduct of a code amount allocated to each frame and an averagequantization scale of the first image compression information is used asthe variable representative of the complexity of the screen for theframe to detect whether or not a scene change is included in the frame,wherein the code amount allocated to each frame of the inputted firstimage compression information includes a code amount at least of abrightness signal component.
 16. An image information conversion methodaccording to claim 15, wherein the code amount allocated to each frameof the inputted first image compression information includes a codeamount regarding a brightness signal component and a code amountregarding a color difference signal component.
 17. An image informationconversion apparatus comprising: conversion means for converting firstimage compression information inputted to said image informationconversion apparatus into second image compression information to beoutputted from said image information conversion apparatus; each of thefirst image compression information and the second image compressioninformation including at least intra-image coded pictures andinter-image predictive coded pictures; and scene change detection meansoperable when said conversion means calculates, based on a variablerepresentative of a complexity of a screen for each frame of theinputted first image compression information, a target code amount foreach frame of the second image compression information to be outputtedfor detecting, prior to conversion of an intra-image coded picture ofthe first image compression information into an inter-image predictivecoded picture of the second image compression information, whether ornot a scene change is included in a frame of the intra-image codedpicture to be converted, wherein, when said scene change detection meansdetects that a scene change is included in the frame to be converted,said scene change detection means limits the conversion by saidconversion means from the intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, and wherein it is detectedwhether or not a scene change is included in the frame to be convertedusing an average value among pixel values of each frame.
 18. An imageinformation conversion method according to claim 17, wherein the averagevalue among the pixels of the immediately preceding intra-image codedpicture is subtracted from the average value among the pixels of theintra-image coded picture, and it is determined that a scene change isincluded when the absolute value of the difference obtained by thesubtraction is higher than a threshold value determined in advance. 19.An image information conversion method according to claim 17, whereinthe average value of the intra-image coded picture among the pixelsincludes an average value at least of a brightness signal componentamong the pixels.
 20. An image information conversion method accordingto claim 19, wherein the average value of pixels of the intra-imagecoded picture includes an average value of the brightness signalcomponent among the pixels and an average value of a color differencesignal component among the pixels.
 21. An image information conversionapparatus comprising: conversion means for converting first imagecompression information inputted to said image information conversionapparatus into second image compression information to be outputted fromsaid image information conversion apparatus; each of the first imagecompression information and the second image compression informationincluding at least intra-image coded pictures and inter-image predictivecoded pictures; and scene change detection means operable when saidconversion means calculates, based on a variable representative of acomplexity of a screen for each frame of the inputted first imagecompression information, a target code amount for each frame of thesecond image compression information to be outputted for detecting,prior to conversion of an intra-image coded picture of the first imagecompression information into an inter-image predictive coded picture ofthe second image compression information, whether or not a scene changeis included in a frame of the intra-image coded picture to be converted,wherein, when said scene change detection means detects that a scenechange is included in the frame to be converted, said scene changedetection means limits the conversion by said conversion means from theintra-image coded picture of the first image compression informationinto an inter-image predictive coded picture of the second imagecompression information, and wherein it is detected whether or not ascene change is included in the frame to be converted using an averagevalue of a DC component among predetermined coding units included ineach frame.
 22. An image information conversion method according toclaim 21, wherein the average value of the DC component among thepredetermined coding units of the immediately preceding intra-imagecoded picture is subtracted from the average value of the DC componentamong the predetermined coding units of the intra-image coded picture,and it is determined that a scene change is included when the absolutevalue of the difference obtained by the subtraction is higher than athreshold value determined in advance.
 23. An image informationconversion method according to claim 21, wherein the average value ofthe DC component among the predetermined coding units of the intra-imagecoded picture includes an average value of the DC component at least ofa brightness signal component among the predetermined coding units. 24.An image information conversion method according to claim 23, whereinthe average value of the DC component among the predetermined codingunits of the intra-image coded picture includes an average value of theDC component of a brightness signal component among the predeterminedcoding units and an average value of the DC component of a colordifference signal component among the predetermined coding units.